Entropy rate of non-equilibrium growing networks 
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New entropy measures have been recently introduced for the quantification of the complexity of 
networks. Most of these entropy measures apply to static networks or to dynamical processes defined 
on static complex networks. In this paper we define the entropy rate of growing network models. This 
entropy rate quantifies how many labeled networks are typically generated by the growing network 
models. We analytically evaluate the difference between the entropy rate of growing tree network 
models and the entropy of tree networks that have the same asymptotic degree distribution. We 
find that the growing networks with linear preferential attachment generated by dynamical models 
are exponentially less than the static networks with the same degree distribution for a large variety 
of relevant growing network models. We study the entropy rate for growing network models showing 
structural phase transitions including models with non-linear preferential attachment. Finally, we 
bring numerical evidence that the entropy rate above and below the structural phase transitions 
follow a different scaling with the network size. 
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I. INTRODUCTION 

Complex networks describe the structure of many bio- 
logical, social and technological systems [IHEj]- Recently, 
new entropy measures have been introduced for the quan- 
tification of complexity of networks ■ Methods for 
quantifying complexity are not only valuable from the 
theoretical point of view, but may lead to important op- 
erational interpretations. This new framework has the 
potential to resolve one of the outstanding problems in 
statistical mechanics of complex systems. In fact, it 
opens the way for a new information theory of complex 
network topologies which will provide an evaluation of 
the information encoded in complex networks. 

The entropy of network ensembles quantifies the num- 
ber of graphs with given structural features such as de- 
gree distribution, degree correlations, community struc- 
ture or spatial embedding 0413] ■ This quantity has been 
shown to be very useful for inference problems defined on 
networks and it has been successfully applied to the prob- 
lem of assessing the significance of features for network 
structure [l5[. Other entropy measures of quantum me- 
chanical nature have been derived by mapping the net- 
work either to a density matrix or to a quantum state 
fl7l - l2l| . These entropies, defined on single networks, set 
a path for the application of tools of quantum informa- 
tion theory to describe the complexity of single networks 
and to introduce new kind of network parameters (for 
example, by considering the notion of correlations and 
subsystems). Finally, entropy rate of random walks (22r - 
|24| are extensively studied on networks and they predict 
how evenly the random walk spreads in the network. For 
many applications indeed one would wish to bias the ran- 
dom walk to construct maximally entropic random walks. 

All these definitions of entropy of networks consider 
static networks, eventually explored by a dynamical pro- 
cess. In this paper we define and evaluate the entropy 



rate of growing network models. The literature in the 
field of growing network models generating scale- free net- 
works is very large [H-13 . By studying the entropy rate of 
these models we aim at quantifying the number of typi- 
cal networks that are generated by these models and we 
compare this number with the number of networks that is 
possible to construct with the same degree distribution. 
In particular we focus on trees to allow for an analytic 
treatment of the problem. Trees are networks in which no 
cycle is allowed. The maximal number of possible tree 
networks generated by a growing network model scales 
like N\ where N is the number of nodes (and links) in 
the network. The minimal number of tree networks gen- 
erated by a growing network model is one, corresponding 
to the formation of a star or of a linear chain. The en- 
tropy rate of growing scale-free networks lies in between 
these two limiting values. Undestanding the value of the 
entropy of graphs is infomative because it describes the 
complexity of the growing network models. In fact the 
value of the entropy will quantify with a unique number 
the size of the space of typical networks generated by the 
growing network model. The smaller is the entropy rate 
of the networks the more complex the network structural 
properties implied by the growing model. In particular 
it is essential to detremine the scaling with N of the en- 
tropy rate, and in the case in which the entropy rate is 
not constant but depends on N it is important to evalu- 
ate the subleading terms that encode for the topology of 
the networks also for other entropy measures [& S E3| • 

The main model of growing scale-free networks is the 
Barabasi- Albert (BA) model 25| that generates scale- 
free networks with power-law exponent 7 = 3. The BA 
networks are known to have weak degree correlations 
due to their causal structure, the growing network model 
with initial attractiveness [26( of the nodes and the fit- 
ness model [13] have more significant correlations. To 
quantify these correlations different measures have been 
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introduced such as the average degree of the neighbor 
of the nodes or the degree correlation matrix. Still we 
lack a way to quantify how much information is encoded 
in growing network models with respect to the networks 
constructed by the configuration model with the same 
degree distribution. 

Here we propose to quantify the number of typical tree 
graphs generated by the non-equilibrium growing net- 
work models [25M3l| as a proxy of their complexity. This 
quantity can be used to measure the fraction of networks 
of given degree sequence that is generated by growing 
network models and to quantify in this way the complex- 
ity of growing network models. Moreover growing net- 
work models as the Bianconi and Barabasi fitness model 
[27l l3(il ] and the non- linear preferential attachment model 
of Krapivsky and Redner [28|, [29| or the growing network 
model with aging of the nodes (3~l| are known to un- 
dergo structural phase transitions as a function of their 
parameters. Here we interestingly see that these phase 
transitions are characterized by a sharp drop of the en- 
tropy rate and strong finite size effects indicating that 
the network is reduced to a more ordered state. 

The remainder of the paper is structured as follows. In 
Section II, we define the Gibbs entropy of networks with a 
given degree distribution. In Section III, we introduce the 
necessary material for studying the entropy rate of grow- 
ing trees. Firstly, we recall the main growing network 
models. Then, we obtain min/max bounds to the en- 
tropy. In Section IV, we study growing trees with station- 
ary degree distribution. In particular, we consider the 
BA model, initial attractiveness, the Bianconi-Barabasi 
fitness model, and networks with structural phase tran- 
sitions. We draw some brief conclusions in Section V. 



II. 



GIBBS ENTROPY OF NETWORKS WITH A 
GIVEN DEGREE DISTRIBUTION 



The Gibbs entropy £ [{£;,;}] [7Hlll] of a network ensem- 
ble with given (graphical) degree sequence {ki} {32l l33j 



n{h}] = j^logAf[{k l }} 



(1) 



where A/"[{fej}] is the number of networks with the spec- 
ified degree sequence and N is the number of labeled 
nodes i = 1,2, ... ,N. The Gibbs entropy depends on 
the number of links but also on the specif details of the 
degree sequence. In Table U we give two illustrative exam- 
ples for two degree sequence compatible with 5 links but 
defining ensembles of networks with different entropy. 

It turns out that the ensemble of networks having a 
given degree distribution is a type of microcanonical 
network ensemble satisfying a large number of hard con- 
straints (the degree of each node is fixed) . It is also pos- 
sible to construct canonical network ensembles similar 
to what happens in classical statistical mechanics when 
one distinguishes the microcanonical and canonical en- 
sembles according to the fact that the energy is perfectly 



Degree 
Sequence 



< -f 



Networks 










Entropy 



£[{**}] = 



E[{fc,}] ^ 



TABLE I: The configuration of networks with degree sequence 
{1,1,1,1,5} (on top, Af[{h}] = 1) and {1,2,2,2,3} (on bottom, 
jV[{h}] = 6). 



conserved or conserved in average. A canonical network 
ensemble with given expected degree sequence is an en- 
semble of graphs in which the degree of each node is dis- 
tributed as a Poisson variable with given expected degree 
{ki}. The entropy of the canonical network ensemble is 
the logarithm of the typical number of networks in the 
ensemble. This entropy >5[{fc,}] is given by 



S[{h}} 



1 

'n 



(2) 

where p^ indicates the probability that a node i is linked 
to a node j. We can evaluate the entropy of a maxi- 
mally random network ensemble with given expected de- 
gree distribuiton {ki} by maximizing the entropy ^[{fci}] 
with respect to under the conditions 



E 



Pij. 



(3) 



In this way we get for the marginal probabilities flpj 

8i0j 



Pi) 



1 



(4) 



where (9; are related to the lagrangian multipliers, or 
"hidden variables" fixed by the constraints given by Eqs. 
([5]). In particular, for the uncorrelated network model in 

the Shannon entropy 



(k)N 



which h < J (k)N and pij 
network ensemble takes a direct form Q 

S[{h}} = i(fc)[log((fc)V)-l]-l^(lnI--l)M5) 

i 

The Gibbs entropy S of a microcanonical ensemble of 
networks with degree sequence {ki} with ki = ki is given 
by [Hi 



nih}} = s[{h}} - n[{ki}} 



(6) 
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where f2[{fcj}] is the entropy of large deviations of the 
canonical ensemble 

mki}] = ~iog[j2p??(i-Pii) 1 - aii n5(j2a ij ,k i K 

a i:j i j 

(7) 

where {caj} is the adjacency matrix of the network. In 
particular the matrix element a,j of the adjacency matrix 
is given by — 1 if a link is present between node i and 
node j while = otherwise. By replica methods and 
the cavity method d, [TTJ] it is possible to derive the given 
expression for r2[{/cj}], 

i 

where Tr r (n) is the Poisson distribution with (n) = r. In 
particular, for the uncorrelated network model in which 
ki < \J (k)N and pij = j^yfy , the Gibbs entropy network 
ensemble takes a direct form Q 

n{ki}} = i(fc)pog«fc)JV)-i]--iX)(k* i -i)* 4 + 

i 

We might as well define the Gibbs entropy EfjA^.}] of 
networks with given degree distribution {Nk}. Since the 
number of graphs with given degree distribution A/"[{iVfc}] 
is just given by 

Af[{N k }}=Af[{h}] T ^ 1 (10) 

Ilk ^k ] - 

it follows that 

n{N k }} = -E^ lo s • ( n ) 



III. ENTROPY RATE OF GROWING TREES 

Many networks are non static but they are growing 
by the addition of new nodes and links. A major class 
of growing networks are growing trees in which at each 
time a new node and a new link is added to the network. 
In the last ten years, many growing network models have 
been proposed. Special attention has been addressed to 
growing network models generating scale-free networks. 
In fact these stylized models explain the basic mechanism 
according to which many growing natural networks de- 
velop the universally found scale free degree distribution. 
The fundamental model for scale-free growing network is 
the BA model [25[ which generates networks with degree 
distribution P(k) ~ fc -7 and 7 — 3. This model is based 
of two ingredients: growth of the network and preferen- 
tial attachment meaning that nodes with large degree arc 



more likely to acquire new links. In this paper we con- 
sider this model and other different significant variations 
to this model including different additional mechanisms 
as initial attractiveness of the nodes [26] . fitness of the 
nodes [13, HTJ] , non-linear preferential attachment (28l. |29| 
and aging of the nodes [3l| . Some of these models as 
explained below undergo structural phase transitions to 
be studied by statistical mechanics methods. 

A. Growing network models 

In the growing scale-free network model we start from 
two nodes linked together, at each time £ = 1,2,... 

• we add a new node i = t + 2; 

• we link the new node to a node i t of the network 
chosen with probability 

n(* t H^ (12) 

where = Eiii A; 

• the number of nodes in the network is N = t + 2. 

As a function of the choice of A,- different networks model 
are defined. In particular we consider the following grow- 
ing network models: 

• If we take Ai = S^i, we get a star graph; 

• If we take Ai = (S^.t+i, we get the linear chain; 

• If we take Ai = 1, we get a maximally random 
connected and growing tree; 

• If we take Ai = ki, where ki is the degree of the 
node i, we get the BA model [25j]; 

• If we take Ai = ki — 1 + a, with a < 1, we get a 
generalized BA model with initial attractiveness of 
the nodes [261 ] : 

• If we assign to each node a fitness value r\i from a 
distribution p(rj) = 1 and rj G (0, 1) and we take 
Ai = rjj k j , we get the Bianconi-Barabasi fitness 
model [271 ]. 

• If we take Ai = kj , we get the non-linear pref- 
erential attachment model of Krapivsky-Redner 
[H, [2t| . This network model undegoes a gelation 
phenomenon for 7' > 1. Namely, there is an emer- 
gence of a single dominant node linked to almost 
every other node. For 7' > 2, there is a finite prob- 
ability that the dominating node is the first node 
of the network. 

• If we assign to each node a fitness value r/i = e~ ,9ei , 
with Cj drawn from a distribution 17(e) oc e K and 
k > 0, and we take Ai = r/iki, we get as a function 
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of f3 the so called " Bose- Einstein condensation in 
complex networks" of Bianconi and Barabasi [3(| ■ 
When this happens, for /3 > f3 c one node with 
high fitness is connected to a finite fraction of other 
nodes in the network. 

If we take Aj = (t — ti)~ a ki where t% indicates the 
time at which the node i has joined the network, we 
get the preferential attachment model with aging 
of the sites of Dorogovtsev-Mendes [3l[. I n this 
growing network the power-law exponent 7 of the 
degree distribution is diverging as 7 ~ ~ when 
a — > l~ . For a > 1 the network is exponential and 
becomes more and more similar to a linear chain. 



B. Entropy rate 

The growing connected trees are fully determined by 
the sequence of symbol X = i 2 , . . . , ijv) where it is 
the node linked at time t to the node i = t + 2. In order 
to evaluate the entropy rate of growing networks it is 
sufficient to determine the entropy rate of the sequence 
(ii,i 2 . . . ,it): 

h(t, X) = P(i t \i 1 ,i 2 , . . . it-i) logP(i t |ii, i 2 , • ■ • , k-i), 

it 

(13) 

where P{i t \i\ 1 i 2 , ■ ■ ■ it—i) is the conditional probability 
that the node it is chosen at time t given the history of 
the process. The entropy of the process evaluating how 
many networks are typically constructed by the growing 
network process is S({X}) 

S({X}) = - ]T P(i 1 ,i 2 ,...i t )logP(i u i 2 ,...,i t ). 

{•81,13, ...fit} 

(14) 



C. Maximal and minimal bound to the entropy 
rate of growing network trees 

It is instructive to study the limits of the entropy rate 
of connected growing trees. The minimal entropy rate is 
given by the entropy rate of the star or of the linear chain. 
Indeed by taking Aj = 5\ i we have that the entropy 
rate is zero. Indeed the growing network model becomes 
deterministic and it gives rise to a unique star network 
with the center on the node i = 1. The entropy rate of a 
linear chain A4 = <5;,t+i is also zero by a similar argument 
and the model generates a unique linear chain network 
structure. On the other hand the maximal entropy rate is 
given by the maximally random growing connected trees 
that is generated by taking A4 = 1 and II; = 1/(4 + 1). 
For this process the entropy rate is given by 



Therefore this entropy rate increases logarithmically 
with time and the probability of each tree with N = t + 2 
nodes is given by 



P(N) = 



1 



(N-l)\ 



(16) 



h(t, X) = Iog(i + 1) 



(15) 



Therefore S({X}) = \og[(N - 1)!]. This is the maximal 
entropy of a growing connected tree. 



IV. GROWING TREES WITH STATIONARY 
DEGREE DISTRIBUTION 

For growing network models with stationary degree 
distributions there are simple relations between h(t, X) 
and S({X}). Indeed let us define the entropy rate 

H{X) = lim hs({X}) - log[(JV - 1)!]. (17) 

iV— >oo iv 

For a growing tree network with stationary degree dis- 
tribution, by the recursive application of chain rule 

P(ii,i 2 ,...,it) = P(it\h,i2,---it-i)P(h,i2,---,it-i) 
we can easily get 

N-2 

H{X)=Ymx [J2h(n,X)-log((N-l)l)]. (18) 

N—yoo iv ' — ' 
n— 1 

If the entropy rate of growing networks H is a constant, it 
means that the number of graphs generated by the grow- 
ing network model has a dominating term which goes like 
Nl and a subleading term that is exponential with the 
number of nodes N. On the contrary if H = —00 it means 
that the number of networks generated by the growing 
network model increases with the number of nodes in the 
network N at most exponentially. Usually the typical 
number of labeled networks generated by growing net- 
work models with convergent degree distribution is less 
than the number of networks with the same degree dis- 
tribution. In order to evaluate the ratio between these 
two cardinalities, we introduce here the difference A be- 
tween the Gibbs entropy £[{iVfc}] of the network with the 
same degree distribution and the entropy of the networks 
generated by the growing network model. Therefore A is 

A = hm \n{N k }} - ^Iog[(JV - 1)!] - H(X) 

"(19) 

The larger the value of A is, the smaller the fraction 
of networks generated by the growing model compared 
with the networks generated by the configuration model. 
This implies that the larger is A the more complex the 
networks generated by the growing models are. In fact 
these networks need the dynamics of the networks im- 
plicitly force the networks to satisfy more stringent set 
of structural conditions beyond the degree distribution. 
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A. The entropy rate of the BA model 



We consider the BA model, we take Ai — ki and II; 
2 (jv-i) therefore 



P(i t \i ll i 2l . . .,it-i) 



2(N - 1) 



(20) 



The BA model, asymptotically in time has a degree dis- 
tribution that converges to the value Nk given by 



N, 



AN 



k(k + l)(k + 2) : 



(21) 



Therefore, asymptotically in time the entropy rate of the 
BA model is 



h(t = N-2,X)^-J2 N k 



loe 



k 



< 2(N-1) °\2(N-1) 

(22) 

Hence, the entropy rate h(X) increases in time as the 
logarithm of the number of nodes in the network, but 
it has a subleading term which is constant in time and 
depends on the degree sequence, i.e. 

h(t = N - 2, X) -> log(JV - 1) + log(2) 
jv-i „ 



fe=i 



(k + l)(k + 2) 



-+ log(JV - 1) - 0.51(0) (23) 

The entropy rate H{X) in the limit N -+ oo is therefore 
given by 



H(X) ~ -0.51 



(24) 



We note here that the degree distribution N k is known 
to have interesting finite size effects 0, [34|, in addition 
to the asymptotic scaling Eq.(|21[), Here we checked that 
the value of the entropy rate is not modified by these 
corrections up to the significant digit we have considered. 
Finally, in order to compare the number of networks gen- 
erated by the BA model with the network that we can 
construct with the same degree sequence, we evaluate 
the value of A in the thermodynamic limit. This can be 
written as 



N-l 



I t=i 
^£jV fc [fclog(fc) + log(27rfc)] 



2N 



1^ — lo S 



N 

k 

0.9(1) 



N 



(25) 



B. The entropy rate of the growing network model 
with initial attractiveness 



If we take Aj = ki — 1 + a, the network generated is 
scale free with power- law exponent 7 = 2 + a (2(| . The 
probability to choose the node it given the history of the 
process is therefore given by 



P(i t \i 1 ,i 2 ,...,it-i) 



1 + a 



Ai 



N (o + l)(JV-l)' 



(26) 



Asymptotically in time the degree distribution for trees 
converges to the value @, [2(| 

N MH-l ^ ^(l + 2 Q )^(fc + a-l) 

N * = N ( 1 + a ) r(a)r ( fc + i + 2a) ' (27) 

From this, the entropy rate hit, X) is asymptotically 



h(t = N— 2, X) -> 

at k-l + a 
~ 2^ Nk ~ TTT777 TT lo § 



k=l 



(a+l)(N-l) 



k-l + a 
(a + l)(A-l) 



. (28) 



which can be simplified as 

h{t = N- 2,X) log(JV - 1) + log(o + 1) 

N-l 



J2 lo § 



k=l 



(k-l + a) 



r(l + 2a)r(fc + a) 



r(o)r(fc + 1 + 2a) 

(29) 



In the limit N — > 00, the entropy rate H(X) is 



fr(*) = iog(o + i) 



N-l 

fc=i 



(k-l + a) 



r(l + 2a)r(A: + a) 

r(o)r(jfe + 1 + 2a) 



.(30) 



When a — > 1, the solution reduces to the solution of the 
BA model. In Fig. Q]we plot the value of H = H(X) ver- 
sus a calculated by Eq. (|30[) using an upper cutoff for the 
degree fc, < KVi = 1, . . . N. As the parameter a — > the 
entropy rate decreases indicating that the network model 
generates an exponentially smaller number of networks. 
Also the Gibbs entropy of scale free networks decreases 
as long as the power-law exponent converges toward 2, 
i.e. in the limit 7 — > 2. In order to evaluate the change in 
the ratio of networks generated by the growing network 
model to the number of possible networks with the same 
degree distribution, in Fig. [2] we plotted A as a func- 
tion of a. As a — ¥ and 7 — > 2 the number of networks 
generated by the growing network models are a smaller 
function of the total number of networks that is possible 
to build with the same degree distribution. This is an 
indication and quantification of the importance of cor- 
relations generated by the growing network model with 
given initial attractiveness a. 
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entropy rate is given by 
h(t = N — 2, X) -> 



rl 



fc=l 



M(iV-l) 



log 



M(iV-l) 



which gives 



-1.59. 



. (34) 



(35) 



FIG. 1: The entropy rate H calculated for the growing net- 
work model with initial attractiveness [2|J as a function of a 
and evaluated by Eq. (|30[) using a maximal degree equal to 
K = 10 7 . 



l 

0.9 
0.8 
0.7 
0.6 
0.5 
0.4 
0.3, 



0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 
a 

FIG. 2: The value of A calculated for the growing network 
model with initial attractiveness [2(| as a function of a evalu- 
ated for networks of N — 50000 nodes and over 20 realizations 
of the process. 



D. Entropy rate for growing network models with 
structural phase transitions 

We have measured the entropy rate for three growing 
network models showing a phase transition: 

• The Krapivsky-Redner model [H HH with 



h(t,X) 



(36) 



and A/" = £li k j 

• The Bianconi-Barabasi model showing a Bose- 
Einstein condensation in complex networks [30j 
with 



h(t,x) = -j2- 



-fob 



loe 



(37) 



C. The entropy rate of the Bianconi-Barabasi 
fitness model 

If the kernel Ai is heterogeneous and specifically given 
by Ai = rjjkjj the model is called Bianconi-Barabasi fit- 
ness model [27|. The probability to choose the node it 
given the history of the process is therefore given by 



P(it\h,i2, 



where p(N — 1) = J^iLi 1 Vi^i- The degree distribution 
iVfe(rj) for nodes of fitness 77, asymptotically in time con- 
verges to m 



/i(iV-l) 



(31) 



N k (v) 



NMv) r(fc)r(i + p/rft 
?] r(k + 1 + n/n) ' 



(32) 



where p(rf) is the distribution of rj. Given the analytic 
solution of the model 0, [27| , /-t is determined by the self- 
consistent relation 



p( V )(jx/r 1 -l)- 1 dr 1 = l 



(33) 



We consider here the case of uniform distribution of the 
fitness, i.e. p(rf) = 1 with 77 G (0,1). Therefore the 



and A/" = £- =1 e^fe* 



• The Dorogovtsev-Mendes model with aging of the 
nodes k5l| with 



h(t,x) = -j2 Ii 



loe 



(38) 



where r, = t — U is the age of node i and M = 

In Fig. [3] the entropy rate H(X) is calculated by numer- 
ical simulations using 



1 



N-2 



H W = x[±^ h{n,X) - log((JV - 1)!)]. (39) 

n— 1 

for a network of sufficiently large size N for the three 
models as a function of the parameters 7, and a re- 
spectively. We show that at the transition point the scal- 
ing of H evaluated for a network of size N changes from 
constant to an N dependent behavior. In particular we 
checked that in the three cases H oc log (AT) indicating 
that as the network grows the typical number of net- 
works that arc generated scales only exponentially with 
N (and not like N\). 
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This behavior signifies a disordered-ordered phase- 
transition in the topology of the network. In the 
Bose-Einstein condensation network model and in the 
Krapivsky-Rcdner model, below the phase transition, the 
network is dominated by a hub node that grabs a finite 
fraction of the nodes. In the aging model, below the 
phase transition, the network develops a structure more 
similar to a linear chain. 



V. CONCLUSION 

In conclusion, we have studied growing network mod- 
els and their entropy rate. We have seen that the entropy 
rate of growing simple trees have maximal and minimal 
bound and we have studied the entropy rate of scale-free 
tree networks. This entropy rate allows us to calculate 
the number of typical graphs generated by growing scale- 
free network models and to quantify their complexity by 
comparing this number to the total number of graphs 



with the same degree distribution. Although we have 
focused on trees the definition of entropy rate can be 
easily extended to growing network models with cycles. 
However the probabilities of adding two or more links 
at a given time should explicetly account for the fact 
that the new links must be distinct, fact which induces 
a small correction to the simple preferential attachment. 
We have analyzed a variety of growing network models 
and we have studied non-equilibrium growing network 
models showing structural phase transitions. By numer- 
ical investigations, we have shown that when a growing 
network model has a phase transition, the entropy rate 
changes its scaling with the system size indicating the 
disorder-to-ordcr transition. In the future, we believe 
that an integrated view of information theory of complex 
networks will provide a framework to extend quantita- 
tive measures of complexity to a large variety of network 
structures, models and dynamics. The present work is a 
step in this direction. 
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FIG. 3: (Color online) The entropy rate H is evaluated for the Kapivsky-Redner model [28|,|29| (panel A), for the " Bose-Einstein 
condesation in complex networks" of Bianconi-Barabasi with g(e) = 2e, and e € (0, 1), (k = 1) [30] (panel B) and for the aging 
model [3J of Dorogovtsev-Mendes (panel C). The data are averaged over N run different realizations of the network . We took 
Nmn = 100 for simulations with JV = 10 4 and N rU n = 30 otherwise. Above the structural phase transition indicated with the 
solid line, the entropy rate H strongly depends on N. 



